Motivation

As discussed in issue #24687, there is a conflict between handle_rpc_request() and Pipeline Parallelism (PP) that causes a deadlock.

Root Cause:

When PP rank 0 receives an RPC request, the following sequence leads to a deadlock:

PP rank 0: recv_requests() → receives RPC request
PP rank 0: process_input_requests() → calls barrier() inside handle_rpc_request()
Other PP ranks: blocked in recv_requests() waiting for requests from rank 0 via point_to_point_pyobj
PP rank 0: hasn't sent requests yet (send happens after process) → DEADLOCK

Modifications

Solution:

Change barrier() in handle_rpc_request() to only synchronize within the TP group instead of globally:

Before

barrier() # Waits for all TP×PP processes

After

barrier(group=self.tp_group.device_group) # Only waits within TP group
This avoids the deadlock because:

The barrier no longer requires cross-stage synchronization (PP ranks don't need to wait for each other)
Each PP stage independently synchronizes its own TP ranks during RPC handling
Other PP ranks can continue executing recv_requests() without being blocked by rank 0's RPC barrier

Accuracy Tests

Speed Tests and Profiling

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Review and Merge Process

1. Ping Merge Oncalls to start the process. See the PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
   • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

CI States

Latest PR Test (Base): ⏳ Run #29631202458
Latest PR Test (Extra): ❌ Run #29631202342